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State Use of ACT—Peer Review Results 


Purpose: 

In order to comply with the regulations put forth by the Elementary and Secondary Education Act (ESEA), as amended by 
the Every Student Succeeds Act (ESSA), states are required to submit evidence associated with their statewide accountability 
systems to the United States Department of Education (ED) for peer review. All states are required to complete the peer 
review process for K-12 statewide assessments in English language arts (ELA), mathematics, science, and English language 
proficiency. Based on the peer review of state submissions, ED has issued decision letters to inform states whether their 
submissions have addressed all requirements. 


Some states have elected to incorporate the ACT assessment into statewide assessment systems, to be used for 
accountability purposes. As of August 2019, seven states identified ACT as a component of their state assessment system 
in their peer review submissions: Montana, Nebraska, North Dakota, Oklahoma, West Virginia, Wisconsin, and Wyoming. 


All seven of these states have received feedback from ED regarding use of the ACT as a statewide assessment, or as an 
assessment option for districts, and whether each state’s peer review submission meets all statutory requirements. The Center 
on Standards and Assessment Implementation (CSAI) reviewed the feedback provided to these seven states to understand 
how utilization of this assessment for accountability purposes meets ED requirements. The information included in this 

brief outlines ED’s feedback to these seven states in the decision letters sent to each state regarding their use of the ACT 
assessment as part of a statewide system. 
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Introduction: 

The assessment peer review process mandated by ESEA, as amended by ESSA, aims to ensure that states implement rigorous 
academic standards and high-quality assessments. The peer review process is evidence-based, drawn from documentation 
submitted by states regarding the process used to develop and administer the assessments, and data-driven to confirm the 
quality of the assessment system. The review is conducted by cohorts of external assessment experts. Since ED paused peer 
review in 2012, almost every state has (a) substantially changed its academic content standards or adopted new ones, and (b) 
adopted new or revised assessments associated with those standards. As a result, all states must submit documentation for 
peer review of reading/language arts, mathematics, and science general and alternate assessments. The peer review process 
must be completed each time a new statewide assessment’ is implemented. 


A State's Guide to the U.S. Department of Education’s Assessment Peer Review Process 


In A State's Guide to the U.S. Department of Education's Assessment Peer Review Process, issued in 2018, ED broke down the type 
of evidence states must submit into 33 Critical Elements (CEs), grouped into seven sections: 


Statewide system of standards and assessments 
Assessment system operations 

Technical quality - validity 

Technical quality - other 

Inclusion of all students 

Academic achievement standards and reporting 
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Locally selected, nationally recognized academic assessments (states may elect to allow districts to administer 
an approved high school assessment different from the statewide high school assessment, though states are 
not required to implement this option) 


+ The State’s Guide document lists each individual CE and includes a description of the level of quality states 
must meet in order to meet the requirements of the ED. 
+ Examples of the types of evidence states may choose to include in their peer review information packet are 
also included for each CE. 
» If a state’s assessment is technology-based, specific examples of evidence are provided as well. 
+ The list of evidence included in the State’s Guide is not exhaustive and states may choose to submit types of 
evidence that are not listed. 
+ The State’s Guide also illustrates which CEs are likely to be addressed by state-specific evidence, which 
are likely to be addressed by evidence that can be coordinated among states that administer the same 
assessments, which may be addressed with a combination of state-specific and coordinated evidence, and 
which CEs will be checked for completeness by ED. 
After a state submits their evidence packets—assembled as instructed by the State’s Guide—teams of expert reviewers 
trained by ED then review them and submit notes on their findings back to states to provide them with initial 
feedback. ED then makes a final determination as to whether submitted evidence illustrates that a state’s assessment 
system either meets, substantially meets, partially meets, or does not meet requirements. If required, states must then 
provide the additional evidence requested to prove that the assessment system(s) in question meets the requirements 
of ED. 


"U.S, Department of Education Peer Review of State Assessment Systems: Non-Regulatory Guidance for States for Meeting Requirements of the Elementary and Secondary 
Education Act of 1965, As Amended. 1st ed. (2015). Washington, DC: U.S. Department of Education. 


?A State's Guide to the U.S. Department of Education's Assessment Peer Review Process. (2018). Washington, DC: U.S. Department of Education. Retrieved from https://www2. 
ed.gov/admins/lead/account/saa/assessmentpeerreview.pdf 
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Peer Review Decisions for States Administering ACT 


The table below outlines ED determinations for state submissions that include ACT as part of the state’s assessment system. This table focuses only on determinations 
related to ACT use for which states need to submit additional state information or documentation to meet statutory requirements for peer review. Not all CEs are 
included in this table. 
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State Summaries: 


Montana 


According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the Montana Office of Public Instruction (OPI) must 


submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


1.4 - Policies for Including All Students in Assessments 


2.1 - Test Design and Development 


2.2 - Item Development 


2.4 - Monitoring Test Administration 


2.5 — Test Security 


3.1 - Overall Validity, including Validity Based on Content 


3.2 - Validity Based on Cognitive Processes 


3.3 - Validity Based on Internal Structure 


4.1 - Reliability 


Evidence of policies that students with disabilities publicly placed in private schools as a means of providing special education 
and related services are included in the assessment system. 


A plan and a timeline to address the alignment issues identified in the existing alignment studies in order to ensure that the 
assessment measures the full range of the State’s academic content standards. 


Strong evidence to establish that OPI’s academic content standards are equivalent to the State cited in the alignment studies. 


Evidence that the State uses reasonable and technically sound procedures to develop and select items to assess student 
achievement based on the State’s academic content standards. 


Evidence for 2.1 will also address this CE. 


Evidence that OP! adequately monitors the administration of the assessments to ensure that test administration procedures are 
implemented with fidelity across Montana school districts and schools. 


Evidence of the detection of test irregularities, remediation following any test security incidents involving any of the State's 
assessments, and investigation of alleged or factual test irregularities. 


Evidence of clear consequences for confirmed violations of test security. 


Evidence that the State’s assessments measure the depth and breadth of Montana's academic content standards. 


Evidence for 2.1 will also address this CE. 


Evidence that the test assesses the cognitive processes found in the depth and breadth of the State’s content standards. 


Evidence for 2.1 will also address this CE. 


Evidence that the scoring and reporting structures of its assessments are consistent with the sub-domain structures of the State's 
academic content standards. 


Evidence for 2.1 will also address this CE. 


Evidence of reliability based on Montana's student population (e.g., reliability estimates for the State overall and for major 
reporting subgroups; standard error of measurement for the State overall and for major reporting subgroups; and estimates of 
classification accuracy and decision consistency for the State overall and for major reporting subgroups—including evidence for 
consistency and accuracy of estimates in categorical classification decisions for the cut scores and achievement levels based on 
assessment results). 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


4.2 - Fairness and Accessibility 


4.7 - Technical Analysis and Ongoing Maintenance 


5.1 — Procedures for Including Students with Disabilities 


5.2 — Procedures for Including English Learners 


5.3 - Accommodations 


5.4 - Monitoring Test Administration for Special Populations 


6.4 — Reporting 


Evidence that the State has taken reasonable and appropriate steps to ensure the assessment is accessible to all students and fair 
across student groups in the State (e.g., analysis of sub-group performance for Montana students on the tests). 


Evidence of an ongoing process for monitoring, maintaining, and improving, as needed, the technical quality of its assessment 
system. 


Provide clear explanation of the differences between assessments based on grade-level academic achievement standards and 
assessments based on alternate academic achievement standards. 


Evidence of the State's process for ensuring that students with disabilities are included in the ACT, with clear guidelines for 
accommodations and the receipt of college-reportable scores. 


Guidance regarding the selection of appropriate accommodations for English Learners. 


Evidence of the State’s process for ensuring that English Learners are included in the ACT with clear guidelines for allowable 
supports and the receipt of college-reportable scores. 


Evidence clarifying that students who receive allowable accommodations will receive equal benefits (e.g., college-reportable 
scores). Evidence submitted for 5.1 may also address this CE. 


Evidence that the State monitors test administration in its districts and schools to ensure that appropriate assessments, with or 
without appropriate accommodations, are selected for students with disabilities under the Individuals with Disabilities Education 
Act, students covered by Section 504, and English learners so that they are appropriately included in assessments and receive 
accommodations that are: 


¢ Consistent with the State's policies for accommodations; 
¢ Appropriate for addressing a student's disability or language needs for each assessment administered; 
¢ Consistent with accommodations provided to the students during instruction and/or practice; 


¢ Consistent with the assessment accommodations identified by a student's IEP team or 504 team for students with disabilities, 
or another process for an English Learner; and 


¢ Administered with fidelity to test administration procedures. 


Evidence that OPI reports to the public its assessment results on student achievement at each proficiency level and the 
percentage of students not tested, for all students and each student group, after each test administration. 


Evidence that OPI follows a process and a timeline for delivering individual student reports to parents, teachers, and principals as 
soon as practicable after each test administration. 
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Nebraska 


According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the Nebraska Department of Education (NDE) must 


submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


1.4 - Policies for Including All Students in Assessments 


2.1 -Test Design and Development 


2.4 - Monitoring Test Administration 


3.1 - Overall Validity, including Validity Based on Content 


3.2 - Validity Based on Cognitive Processes 


3.3 - Validity Based on Internal Structure 


5.1 — Procedures for Including Students with Disabilities 


5.2 - Procedures for Including English Learners 


5.3 —- Accommodations 


Evidence that the State clearly and consistently communicates to districts and schools the requirement that all public elementary and secondary 
school students must participate in assessments. 


Evidence of policies stating that all students with disabilities publicly placed in private schools, as a means of providing special education and 
related services, must be included in the assessment system. 


Evidence that the test design is aligned to the depth and breadth of the State’s high school academic content standards (e.g., evidence of 
alignment of the test design blueprint to academic content standards). 


Evidence that NDE adequately monitors the administration of its State assessments to ensure that standardized test administration procedures 
are implemented with fidelity across districts and schools. 


Documentation of adequate alignment between the State’s assessments and the State’s academic content standards that the assessments are 
designed to measure in terms of content (i.e., knowledge and process), the depth and breadth of the State’s academic content standards, balance 
of content, and cognitive complexity. 


Evidence that NDE’s assessment items (and item types) tap the intended cognitive processes appropriate for each grade level as represented in 
the State’s academic content standards. 


Evidence that the scoring and reporting structures are consistent with subdomain structures of the State’s academic content standards on which 
the intended interpretations and uses of results are based (such as a factor analysis). 


Evidence of the State's process for ensuring that students with disabilities are included in the ACT, with clear guidelines for accommodations and 
the receipt of college-reportable scores. 


Evidence that students with disabilities are not denied the opportunity to participate in the assessment or denied any benefits from participation 
in the assessment. 


Evidence of the State’s process for ensuring that English Learners are included in the ACT with clear guidelines for allowable supports and the 
receipt of college-reportable scores. 


Evidence that students with disabilities are not denied the opportunity to participate in the assessment or denied any benefits from participation 
in the assessment.. 


Evidence demonstrating that the accommodations provided: 
¢ Are appropriate and effective for meeting the individual student’s need(s) to participate in the assessments; 
¢ Do not alter the construct being assessed; and 


¢ Allow meaningful interpretations of results and comparison of scores for students who need and receive accommodations as well as for 
students who do not need and do not receive accommodations. 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


5.4 - Monitoring Test Administration for Special — Evidence that the State monitors test administration in districts and schools to ensure that appropriate assessments, with or without appropriate 
Populations accommodations, are selected for students with disabilities under the Individuals with Disabilities Education Act, students covered by Section 
504, and English learners, to ensure that accommodations are administered with fidelity and State test administration procedures are followed. 


6.3 - Challenging and Aligned Academic Achievement - Evidence that the State’s academic achievement standards are challenging and aligned to the State’s academic content standards. . 
Standards 
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North Dakota 


According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the North Dakota Department of Public Instruction 
(NDDPI) must submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


2.1 - Test Design and Development - Evidence that the ACT is aligned to the North Dakota academic content standards (e.g., an alignment study of the ACT with the State’s academic 
content standards). 


- Evidence of a test design for the ACT that includes a balance of depth of knowledge (DOK) across and within reporting categories that is 
representative of the cognitive demand found within the State’s academic content standards. 


2.4 - Monitoring Test Administration - Evidence of a process for monitoring the administration of the ACT to ensure that standardized test administration procedures are implemented 
with fidelity across districts and schools. 


3.1 — Overall Validity, including Validity Based on Content - Evidence that all standards are included in the assessment system, including any standards that are excluded because they are not reading/ 
language arts “Anchor Standards” or “standards for mathematical practice.’ 


- Evidence for 2.1 will also address this CE. 


3.2 - Validity Based on Cognitive Processes - Evidence of validity that the ACT assessment taps the intended cognitive processes as represented in the State’s content standards (e.g., think- 
aloud labs, item analysis protocols, and surveys following test items). 


3.3 - Validity Based on Internal Structure - Evidence of internal structure validity of the ACT writing test (e.g., correlations among domains scores). 
4.1 - Reliability - Evidence that the issue of lower ACT test reliability for North Dakota Native Americans and Alaskan Natives was addressed to determine possible 
causes. 


- Evidence of reliability, overall standard errors of measurement (SEM), and conditional SEM of subtests for the ACT writing test. 


4.7 - Technical Analysis and Ongoing Maintenance - Evidence of a process (i.e., including the State's technical advisory committee (TAC) for monitoring, maintaining, and improving the State's 
administration of the ACT [e.g., TAC meeting agendas and minutes)). 


- Evidence of how adequate technical quality will be made public on the State's website 


5.2 — Procedures for Including English Learners (ELs) — Evidence that all of the supports and accommodations available to English Learners on the State’s high school assessment (NSDA) are also 
available to students in a local education agency (LEA) that participates in the ACT as a locally selected option. 


5.3 — Accommodations — Evidence that the accommodations provided do not alter the construct being assessed and allow for meaningful interpretations of results. 


- Evidence that the use of any non-allowable accommodations on the test will result in an invalid individual score result. 


5.4 - Monitoring Test Administration for Special - Evidence of a plan and forms for collecting information (e.g., monitoring of classrooms, self-report by teachers) linking the accommodations 
Populations actually used during classroom instruction with the accommodations in the IEP and the ones used during testing. 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


6.3 - Challenging and Aligned Academic Achievement 
Standards 


6.4 — Reporting 


7.1 — State Procedures for the Use of Locally Selected, 
Nationally Recognized High School Academic Assessments 


7.3 - Comparability of the Locally Selected, Nationally 
Recognized High School Academic Assessments with the 
State Assessments 


Evidence that the State’s academic achievement standards for the ACT test in reading/language arts are challenging and aligned to the State's 
academic content standards and with the entrance requirement for credit-bearing coursework in the system of public higher education in the 
State such that a student who scores at the proficient or above level has mastered what students are expected to know and be able to do by the 
time they graduate from high school in order to succeed in college and the workforce. 


Evidence that the student reports: 
¢ Report the student’s academic achievement in terms of the State’s grade-level academic achievement standards; and 
¢ To the extent practicable, are available in alternative formats (e.g., oral or written translations and accessible formats as needed). 


¢ Evidence of the reliability of each of the subscale scores on score reports, or the addition of confidence intervals reflecting the level of 
precision. 


Evidence that the State has established technical criteria to use in its review of the ACT as a locally selected, nationally recognized high school 
academic assessment. 


Evidence that the State has completed its technical review of the ACT. 
Evidence provided for CEs 2.1, 2.3, 5.2, and 5.3 will also address this CE. 


Evidence of comparability between the ACT tests and the North Dakota State Assessment tests (e.g., tables comparing reliability and validity 
coefficients for the tests). 


Evidence provided for CEs 2.1 and 3.1 will also address this CE. 
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Oklahoma 


According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the Oklahoma State Department of Education 
(OSDE) must submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


1.5 - Meaningful Consultation in — Evidence that the State has conducted meaningful and timely consultation with representatives of American Indian tribes located in the State in developing the academic 


the Development of Challenging 
State Standards and Assessments 


2.1 - Test Design and 
Development 


3.1 - Overall Validity, including 
Validity Based on Content 


3.2 - Validity Based on Cognitive 
Processes 


3.3 - Validity Based on Internal 
Structure 


4.1 - Reliability 


4.7 - Technical Analysis and 
Ongoing Maintenance 


5.1 - Procedures for Including 
Students with Disabilities 


content standards adopted in 2016. 


¢ Evidence that the test design of the ACT addresses the full depth and breadth of the State’s academic content standards, such as test blueprints which indicate the 
number or percentage of items by depth of knowledge (DOK) classification within each reporting category of the academic content standards. 


Documentation of adequate alignment between the ACT and the full breadth and depth of the State’s academic content standards that its assessments are designed to 
measure, specifically that: 


¢ The ACT assesses all of the academic content standards in reading/language arts. 


¢ The ACT reporting categories align with the State’s academic content standards (e.g., there is a coherent relationship between the State's mathematics standards and 
the mathematics standards represented by the ACT’s reporting categories). 


¢ The State has confirmed that all planned changes/updates to ACT forms based upon findings of the alignment evaluation have been implemented. 


Evidence of validity that the ACT taps the intended cognitive processes as represented in the State’s content standards (e.g., think-aloud items, item analysis protocols, and 
surveys following test items). 


Adequate validity evidence that the scoring and reporting structures of the ACT writing test are consistent with the sub-domain structures of the State’s content standards 
(e.g., correlations among domain scores). 


Evidence of subtest reliabilities. 
Evidence for overall and conditional standard error of measurement (CSEM) by student group. 


Evidence of reliability, including overall standard error of measurement (SEM) and CSEM of subtests for the ACT writing test. 


Evidence that OSDE has a system for monitoring, maintaining, and improving, as needed, the quality of its assessment system, including clear and technically sound 
criteria for the analyses of the ACT. 


Evidence that decisions about how to assess students with disabilities are made by a student's IEP team under the Individuals with Disabilities Education Act, the 
placement team under Section 504, or the individual or team designated by a district to make that decision under Title Il of the ADA, as applicable, based on each 
student's individual abilities and needs. Specifically, evidence that the State has determined what accommodations are allowable on the ACT and communicates the 
allowable accommodations to IEP teams. If IEP teams select allowable accommodations for the State tests, including the ACT, then the student must be allowed to take 
the test with those accommodations and receive a valid score. 


Evidence that no student with a valid score on the State assessment (including the ACT) is denied the equal benefits of other students (i-e., all students with a valid score 
receive a college-reportable score). 


5.2 — Procedures for Including 
English Learners (ELs) 


5.3 —- Accommodations 


6.3 - Challenging and Aligned 
Academic Achievement 
Standards 


6.4 — Reporting 


7.1 - State Procedures for the Use 
of Locally Selected, Nationally 
Recognized High School 
Academic Assessments 


7.2 - State Monitoring of Districts 
Regarding the Use of Locally 
Selected, Nationally Recognized 
High School Academic 
Assessments 


7.3 - Comparability of the Locally 
Selected, Nationally Recognized 
High School Academic 
Assessments with the State 
Assessments 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


Evidence of procedures for determining whether an English Learner should be assessed with linguistic accommodation(s). Specifically, evidence that the State has 
determined what EL accommodations are allowable on the ACT and communicates the allowable accommodations to LEAs. If EL service teams in LEAs select allowable 
accommodations for the State tests, including the ACT, then the student must be allowed to take the test with accommodations and receive a valid score. 


Evidence that no student with a valid score on the State assessment (including the ACT) is denied the equal benefits of other students (i-e., all students with a valid score 
receive a college-reportable score). 


Evidence of a process to individually review and allow exceptional requests for a small number of students who require accommodations on the ACT beyond those 
routinely allowed. 


Evidence that the accommodations provided do not alter the construct being assessed and allow for meaningful interpretation of results. 


Evidence that OSDE ensures that accommodations for the ACT do not deny students with disabilities or English Learners the opportunity to participate in the assessment 
or any benefits from participation in the assessment (see the evidence requested in CEs 5.1 and 5.2 regarding allowable accommodations for students with disabilities and 
English Learners, respectively). 


Evidence that the ACT achievement standards for mathematics and reading/language arts align with entrance requirements for credit-bearing coursework in the system 
of public higher education in the State and relevant State career and technical education standards such that a student who scores at the proficiency or above level has 
mastered what students are expected to know and be able to do by the time they graduate from high school in order to succeed in college and the workforce. 


Evidence that student reports: 

¢ Provide information to help parents, teachers, and principals interpret the test results and address the specific academic needs of students; and 
¢ To the extent practicable, are available in alternative formats (e.g., oral or written translations and accessible formats as needed). 

Evidence of the reliability of each of the subscale scores on score reports, or the addition of confidence intervals reflecting the level of precision. 
Evidence of how the state will report mathematics scores consistent with the model it validated through confirmatory factor analysis. 


Evidence submitted for CE 3.3 will also address this CE. 


Evidence that the State has established technical criteria to review any selection of a nationally recognized high school assessment. 


Evidence that the State has completed this review of the ACT using its established technical criteria and has found that the use of the locally selected assessment meets its 
criteria. 


Evidence requested for CEs 5.1, 5.2, and 5.3 is needed to also address this CE in terms of allowable accommodations. 


Evidence that OSDE has procedures to ensure that before an LEA requests approval to use a nationally recognized assessment like the ACT, it describes how the LEA 
notified all parents of high school students it serves: 


¢ That the LEA intends to request approval from OSDE to use a nationally recognized high school academic assessment in place of the statewide academic assessment; 


¢ How parents, and as appropriate, students may provide meaningful input regarding the district's request (includes students in public charter schools who would be 
included in such assessments); and 


¢ Of any effect of such a request on the instructional program in the LEA. 


Evidence of comparability between the ACT test and the statewide test (the SAT) (e.g., tables comparing reliability coefficients for the tests, in addition to correlations 
between the two tests already provided). 


Evidence for CEs 2.1 and 3.1 are needed to address this CE. 
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West Virginia 
According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the West Virginia Department of Education (WVDE) 
must submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


1.2 - Coherent and Rigorous Academic Standards 


2.1 -Test Design and Development 


3.1 — Overall Validity, including Validity Based on Content 


3.2 - Validity Based on Cognitive Processes 


4.1 - Reliability 


4.7 - Technical Analysis and Ongoing Maintenance 


5.2 - Procedures for Including English Learners (ELs) 


5.3 - Accommodations 


6.1 - State Adoption of Academic Achievement Standards 
for All Students 


6.2 - Achievement Standards Setting 


Evidence that the State's challenging academic content standards are aligned to entrance requirements for credit-bearing coursework in the 
State’s system of higher education and relevant State career and technical standards. 


¢ Evidence that the ACT is aligned to the West Virginia academic content standards (e.g., an alignment study of the ACT with the State’s 
academic content standards). 


¢ Evidence of the test design for the ACT that includes a balance of the cognitive demand (as expressed through depth of knowledge that is 
representative of the cognitive demand found within the State’s academic content standards). 


Evidence that the content of operational forms of the ACT is aligned to the State’s academic content standards. 
Evidence for CE 2.1 will also address this CE. 


Evidence of validity that the ACT taps the intended cognitive processes as represented in the State’s content standards (e.g., think-aloud items, 
item analysis protocols, and surveys following test items). 


Given the low reliability estimates for many subscale scores, if the State chooses to report subscales in each content area, WVDE must report the 
subscales with confidence intervals. 


Reliability and overall CSEM by student group are needed. 


Evidence that WVDE makes information about the technical quality of the ACT public, including on the State’s website. 


Evidence that all of the supports and accommodations available to English Learners on the State's high school assessment (the SAT) are also 
available to students in an LEA that participates in the ACT as a locally selected option. 


Evidence that the accommodations provided on the ACT do not alter the construct being assessed and allow for meaningful interpretation of 
results. 


Evidence that the use of any non-allowable accommodations on the ACT will result in an invalid individual score result. 


Evidence that the State has formally adopted academic achievement standards for the ACT science test and applies these standards to all grades 
to which they apply. These standards must have at least three levels of achievement, descriptions of competencies associated with each, and 
scores differentiating among the levels. 


Evidence of a technically sound method and process that involved panelists with appropriate experience and expertise for setting academic 
achievement standards for the ACT science test that are comparable with standards from the State's high school science assessment. 


Evidence to support the establishment of achievement standards for the State’s high school tests (the SAT) so that the comparability of the ACT's 
concordant achievement standards can be evaluated. 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


6.3 - Challenging and Aligned Academic Achievement 
Standards 


6.4 — Reporting 


7.1 — State Procedures for the Use of Locally Selected, 
Nationally Recognized High School Academic Assessments 


7.3 - Comparability of the Locally Selected, Nationally 
Recognized High School Academic Assessments with the 
State Assessments 


Evidence of cut scores and achievement level descriptors on the ACT for all content areas (reading/language arts, mathematics, and science). 


As noted in CE 6.2, evidence to support the establishment of achievement standards for the State's high school tests (the SAT) so that the 
comparability of the ACT's concordant achievement standards can be more readily evaluated. 


Evidence that student reports: 

¢ Include the student's academic achievement in terms of the State’s academic achievement standards; 

¢ Provide information to help parents, teachers, and principals interpret the test results and address the specific academic needs of students; 
¢ Are provided in an understandable and uniform format; and 


¢ To the extent practicable, are available in alternative formats (e.g., oral or written translations and accessible formats as needed). 


Evidence that student reports: 

Include the student's academic achievement in terms of the State’s academic achievement standards; 

Provide information to help parents, teachers, and principals interpret the test results and address the specific academic needs of students; 
Are provided in an understandable and uniform format; and 


To the extent practicable, are available in alternative formats (e.g., oral or written translations and accessible formats as needed). 


Evidence of comparability between the ACT and the statewide high school assessments (the SAT) (e.g., tables comparing reliability coefficients 
for the tests, in addition to correlations between the two tests already provided). 


Evidence requested for CEs 2.1, 3.1, and 6.2 are also needed to address this CE. 


State Use of ACT—Peer Review Results 


Wisconsin 
According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the Wisconsin Department of Public Instruction 


(WI DPI) must submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


2.1 - Test Design and Development - Evidence that the test design addresses the full breadth and depth of the academic content standards, specifically that gaps identified in the 
alignment study have been addressed and implemented on the operational form of the assessment. 


3.1 - Overall Validity, including Validity Based on Content - Evidence of adequate validity based on test content, specifically that gaps in the test content, as identified in the alignment study, have been 
incorporated on operational forms of the assessment. 


4.1 - Reliability - Evidence that the State has documented adequate reliability evidence for its assessments for the following measures of reliability for each 
student group consistent with nationally recognized professional and technical testing standards, specifically the analysis and interpretation of 
potential reasons for lower reliability estimates of test scores for English learners in all subjects. 


State Use of ACT—Peer Review Results 


Wyoming 


According to peer review feedback, to meet all peer review requirements for the use of ACT in its state assessment system, the Wyoming Department of Education (WDE) 


must submit the following documentation: 


CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


2.1 —-Test Design and Development 


2.3 —Test Administration 


2.4 - Monitoring Test Administration 


2.5 — Test Security 


2.6 — Systems for Protecting Data Integrity and Privacy 


3.1 - Overall Validity, including Validity Based on Content 


3.2 — Validity Based on Cognitive Processes 


3.3 - Validity Based on Internal Structure 


4.2 - Fairness and Accessibility 


5.1 - Procedures for Including Students with Disabilities 


5.2 - Procedures for Including English Learners (ELs) 


5.3 - Accommodations 


Documentation of independent alignment studies between the State’s academic content standards and the State’s high school assessment (the 
ACT). 


Evidence of State policies for test administration to address unexpected or irregular testing situations that may arise during the course of test 
administration. 


Documentation that clearly identifies or clarifies WDE’s role in the monitoring of the ACT test administration. 


Documentation outlining what remediation WDE requires following test security violations. 


Documentation of a test security agreement with ACT, including procedures ACT follows to report incidents to WDE. 


Evidence of a specific data security agreement between WDE and ACT. 


Evidence of independent alignment studies evaluating the test items to the State content standards for all assessments. 


Documentation that all of the State’s assessments measure the intended cognitive processes appropriate for each grade level as represented in 
the State's academic content standards. 


Evidence that the reporting structures are consistent with the sub-domain structures of the State’s academic content standards. 


Documentation on the accommodations that can be used for each assessment, grade, and subject area. 


Training materials given to ACT fairness review committees and item writers, including demographic information on these reviewers. 


Documentation of training provided to teachers on accommodations. 


Documentation to clarify the relationship between WDE's inclusion/accommodation policies and ACT's. 


Evidence of training for teachers on EL accommodations. 
Evidence that English Learners receive appropriate accommodations related to their English proficiency based on their individual needs. 
Documentation of the process, including who is on the decision-making team, for determining accommodations for English Learners. 


Evidence of a decision-making framework that schools use to make accommodation decisions. 


Documentation of the process for reviewing an exceptional accommodation request. 
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CRITICAL ELEMENT ADDITIONAL DOCUMENTATION NEEDED 


5.4 - Monitoring Test Administration for Special 
Populations 


6.1 - State Adoption of Academic Achievement Standards 
for All Students 


6.2 — Achievement Standards Setting 


6.4 — Reporting 


Evidence that WDE monitors test administration in its districts and schools to ensure that appropriate assessments, with or without appropriate 
accommodations, are selected for students with disabilities under the Individuals with Disabilities Education Act, students covered by Section 
504, and English Learners so they are appropriately included in assessments and receive accommodations that are: 


¢ Consistent with the State's policies for accommodations; 
¢ Appropriate for addressing a student's disability or language needs for each assessment administered; 
* Consistent with accommodations provided to the students during instruction and/or practice; and 


¢ Consistent with the assessment accommodations identified by a student's IEP team or 504 team for students with disabilities, or another 
process for an English Learner. 


Administered with fidelity to test administration procedures. 


Documentation of formal adoption of academic achievement standards for high school. 


Clarify how the WDE performance-level descriptors for the ACT are connected to WDE's content standards and to the interpretations of academic 
knowledge and skills. 


Documentation of the availability of score reports in alternate formats (e.g., Braille, translations, etc.). 


Documentation of the process and timeline for delivering individual score reports. 


